Skip to content

Keep MTP sessions alive after uncorrelated JSON-RPC errors - #12283

Merged
Amaury Levé (Evangelink) merged 3 commits into
mainfrom
dev/amauryleve/rider-protocol-investigation
Nov 9, 2026
Merged

Amaury Levé (Evangelink) merged 3 commits into
mainfrom
dev/amauryleve/rider-protocol-investigation

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Pull request

Summary

Keep Microsoft.Testing.Platform server-mode sessions alive when a peer sends a valid JSON-RPC error response with an explicit null ID. Fixes #12261.

The error model, both JSON engines, the separately shipped source-only client, and the protocol schema now distinguish an uncorrelated error from a missing or invalid ID. Such errors are diagnosed without replying, completing an unrelated request, or adding work to the host's shutdown counter. Normal numeric/canonical-string request, response, and cancellation IDs retain their existing behavior.

This does not recover Rider's lost messages, change UTF-8 framing, or implement general malformed-message recovery. A servicing backport is being prepared for rel/4.5 / MTP 2.5.3.

Design and decision record

The issue investigation and scoped decision is the maintained rationale for this bug fix; no new feature/design document is necessary. JSON-RPC 2.0 section 5 requires null when the request ID cannot be determined.

Independent GPT-6.1 Sol and Claude Sonnet 5 plan reviews agreed on the narrow fix, with safeguards incorporated here: error-only null support, strict null-error envelopes, balanced shutdown accounting, no guessed request correlation, and real packaged modern/Framework coverage. We deliberately did not relax shared ID helpers, introduce capabilities/public APIs/timeouts, or promise stream resynchronization.

Demonstration

The previously reproduced shipping MTP 2.5.2 net48 sequence was:

initialize succeeds
receive {"jsonrpc":"2.0","id":null,"error":{"code":-32700,"message":"JSON parse error"}}
unhandled MessageFormatException: 'id' field is missing
process exit -532462766

The actual packed-server regressions now send the same literal frame repeatedly between completed RPC requests, discover exactly one action node (uid=TestMethod1, display-name=TestMethod1, state discovered), execute that same node with state passed, and verify server exit 0. They compare the executed DLL bytes to the selected fresh NuGet asset and verify the server's reported loaded assembly path. The packed source-client consumers also decode the literal null-ID error before launching and driving a real server.

This is a nonvisual protocol change, so a readable sequence and executable assertions are more useful than screenshots. Rider itself and its reported Unicode parser failure were not available for capture; IDE recovery remains unverified, not implied by the passing wire tests.

Validation and regression proof

Revision: 9ab03e7774cb99667362e88e0157dc2d10e89fb1, based on 127798de9bd5c3f7204dc8932cddf043bf4bd6c6. The initial full-pack/18-case run used the identical source as that base plus patch SHA-256 F576DC76E121E2F87E96432F662AB59DD9825A4DFFF1CBF8CA7676BF4727CF39; the paired restored-implementation runs rebuilt and repacked the committed revision itself.

Toolchain: repository-pinned .NET SDK 12.0.100-alpha.1.26509.101; Debug configuration. Unit engines ran on net10.0, net11.0, and Windows net462 (Jsonite). The outer acceptance host was net12.0; generated shipping consumers exercised net10.0, net11.0, and Windows net48, including the packed C# 12/down-level source slice. The full Windows pack used the repository's desktop MSBuild path because MSTest's UAP support requires it.

The following are the executed command shapes; $evidenceDir and $results were task-owned, separate directories, with unique binary-log names retained for every build/test/pack:

.\build.cmd -pack -c Debug -msbuildEngine vs -projects TestFx.slnx "/bl:$binlog" /nr:false

.\.dotnet\dotnet.exe test --project test\UnitTests\Microsoft.Testing.Platform.UnitTests\Microsoft.Testing.Platform.UnitTests.csproj -c Debug -f $tfm --no-build --filter "FullyQualifiedName~FormatterUtilitiesTests|FullyQualifiedName~ServerTestHostTests" -p:UsingDotNetTest=true "-p:ArtifactsTestResultsDir=$results" "-bl:$evidenceDir\green-core-$tfm-test-{}.binlog" --show-test-results failed

.\.dotnet\dotnet.exe test --project test\UnitTests\Microsoft.Testing.Platform.ServerMode.Client.Sources.UnitTests\Microsoft.Testing.Platform.ServerMode.Client.Sources.UnitTests.csproj -c Debug -f $tfm --no-build --filter "FullyQualifiedName~MtpServerClientTests|FullyQualifiedName~SerializationMutationTests" -p:UsingDotNetTest=true "-p:ArtifactsTestResultsDir=$results" "-bl:$evidenceDir\green-client-$tfm-test-{}.binlog" --show-test-results failed

.\.dotnet\dotnet.exe test --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj -c Debug --no-build --filter "(FullyQualifiedName~MtpServerClientPackagedConsumerRunTests&FullyQualifiedName!~PackagedNet6)|FullyQualifiedName~MtpServerClientSourcePackageConsumerTests|FullyQualifiedName~MtpServerClientSourcePackageTests" -p:UsingDotNetTest=true -p:TestRunnerAdditionalArguments= "-p:ArtifactsTestResultsDir=$results" "-bl:$evidenceDir\green-acceptance-complete-pack-{}.binlog" --show-test-results all
Command / selection Result
Complete configuration-matched pack Exit 0; zero warnings/errors; package-content/metadata guard passed.
Core unit selection, net10.0 / net11.0 / net462 164 / 164 / 164 executed and passed; exit 0 each; no skips.
Source-client unit selection, net10.0 / net11.0 / net462 279 / 279 / 217 executed and passed; exit 0 each; no skips.
Independently discovered packaged selection Planned 18, selected 18, executed 18, passed 18, failed/skipped 0; outer exit 0.
Repository verify_execution.py on the acceptance TRX and source/discovery-derived expected names Exit 0; verified freshness, identities, counts, and outcomes.
JSON Schema validation using jsonschema.Draft202012Validator 15 positive/negative envelope and ID-boundary cases passed; exit 0.
Git diff and encoding checks Clean whitespace; every changed C# file retains UTF-8 BOM.

Requirement coverage

Requirement Evidence
Accept a genuine uncorrelated error and preserve payload Deserialize_NullIdError_PreservesUncorrelatedPayload, Serialize_NullIdError_EmitsExplicitNullId; both engines.
Do not widen other ID/envelope rules Deserialize_NullIdError_RejectsInvalidEnvelope, Deserialize_CorrelatedError_PreservesIdKind, Deserialize_MissingErrorId_RemainsInvalid, Deserialize_NullSuccessId_RemainsInvalid, Deserialize_NullCancellationId_RemainsInvalid, existing NullRequestId_IsRejected.
No reply/count leak or diagnostic-sink failure RunAsync_NullIdErrorsAreDiagnosedWithoutLeakingRequestCounts with normal and throwing loggers.
Preserve pending response/cancellation/EOF/disposal and ID zero RunTestsAsync_NullIdErrors_PreservePendingRequestLifecycle, JsonRpcConnection_NullIdError_DoesNotCompleteRequestZero.
Real packed server survives and reports the exact same node correctly PackagedServer_NullIdErrorsPreserveDiscoveryExecutionAndExit on both modern TFMs; PackagedFrameworkServer_NullIdErrorsPreserveDiscoveryExecutionAndExit on net48.
Packed source client, not just a project reference PackagedConsumer_LaunchesRealServer_DiscoversAndRunsExpectedNode, PackagedFrameworkConsumer_NullIdErrorIsAcceptedAndRealServerRemainsUsable, HostileConsumer_CompilesAgainstPackedSource, and all selected source-package anti-drift guards.

The test assemblies retain method-level scheduling. New mutable state, loopback peers, generated assets, and source-consumer caches are per test; no assembly opt-in/opt-out or global test lock was changed. Windows release CI selects the ordinary solution tests when SkipTests is false; live CI execution is not claimed by local validation.

Test-value evidence

Executed: detected. In a separate, task-owned sibling worktree, only the two error deserializers were restored to their pre-fix strict-ID behavior from 127798de9bd5c3f7204dc8932cddf043bf4bd6c6. The nullable model/API, writers, dispatch safeguards, test code, assertions, scheduling, and selection were retained. This is a buildable targeted counterfactual, not a pristine baseline.

Counterfactual patch SHA-256: 4BC19D9A3E4FF3C185982BF2A72E1CF5C4E7B0979F8737E6B45B1230B5878CB5. Both variants were rebuilt/repacked separately and generated consumers restored into fresh caches.

Contract / test case Without the fix Correct implementation
Deserialize_NullIdError_PreservesUncorrelatedPayload 3 cases on net10.0 and 3 on net462 executed/failed with the intended MessageFormatException; test exits 2. Identical 6 executions passed; exits 0.
The six packaged server/source-client cases named above All 6 executed/failed: packed clients rejected the literal null-ID envelope, and initialized packed servers lost the connection after receiving it; outer exit 2. Identical 6 cases passed after restoring/rebuilding/repacking commit 9ab03e7774cb99667362e88e0157dc2d10e89fb1; outer exit 0, fresh TRX identity/count gate passed.
Correlation, shutdown, logger-failure, invalid-envelope guards The strict-ID counterfactual does not separately mutate these safeguards; no separate mutation score is claimed. Focused 1,267-case unit runs passed, including all 102 new regression executions across the three TFMs.

Paired selectors were FullyQualifiedName~Deserialize_NullIdError_PreservesUncorrelatedPayload and FullyQualifiedName~MtpServerClientPackagedConsumerRunTests&FullyQualifiedName!~PackagedNet6. The isolated source was restored clean afterwards. Retained evidence includes TRX/JUnit/CTRF results, unique binlogs, source/counterfactual patches, package hashes, and resolved/loaded asset paths. Local artifact paths are intentionally not presented as publicly accessible CI links.

Remaining gap: no real Rider installation/traffic capture or IDE rendering/recovery validation. General malformed framing remains outside this PR.

Accept explicit null IDs only for genuine error responses. Preserve request correlation and shutdown accounting across the platform and source-only client, with modern and Framework packaged regressions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 9, 2026 12:50
@github-actions github-actions Bot added the state/needs-review Awaiting review from the team. label Nov 9, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧪 Test quality review — PR #12283

Compared merge base 127798de9bd5c3f7204dc8932cddf043bf4bd6c6 with head 9ab03e7774cb99667362e88e0157dc2d10e89fb1. Evidence applies to this head, not every intermediate commit. This review is static; no tests or builds were run.

Test-value evidence

Contract Scenario / expected outcome Decisive test and assertion Counterfactual Evidence status
A valid error with id:null is accepted as an uncorrelated ErrorMessage and its payload is preserved (both JSON engines) Code is -32700, -32600 or -31001. Id and StringId are null, and code, message and data are preserved. Deserialize_NullIdError_PreservesUncorrelatedPayload: Assert.IsNull(error.Id) plus exact code, message and data assertions. Pre-fix strict-ID deserializer throws MessageFormatException. Static prediction: would be detected. The PR description reports an executed paired run (6 cases failed without the fix, passed with it). I could not access those records, so they are unverified here.
The serializer emits an explicit "id":null ErrorMessage(null, ...) serializes with id present and null. Serialize_NullIdError_EmitsExplicitNullId: exact JSON equality. Reverting the (object?) cast or the nullable model would drop or break the id. Static prediction: would be detected.
Strict envelope validation is unchanged for everything else Method or result next to a null id, malformed error or code or message, missing id on an error, null id on a success response, and null cancel id are all rejected. Correlated numeric and string ids keep their kind. Deserialize_NullIdError_RejectsInvalidEnvelope (13 cases), Deserialize_CorrelatedError_PreservesIdKind, Deserialize_MissingErrorId_RemainsInvalid, Deserialize_NullSuccessId_RemainsInvalid, Deserialize_NullCancellationId_RemainsInvalid. Relaxing id handling or the envelope checks would make ThrowsExactly<MessageFormatException> fail. Static prediction: would be detected.
The server host diagnoses a null-id error without replying or leaking the shutdown request counter Two null-id errors, then Initialize, then Exit. The server replies only once, exits 0, and logs two diagnostics, even when the logger throws. RunAsync_NullIdErrorsAreDiagnosedWithoutLeakingRequestCounts: Assert.AreEqual(0, run), Times.Once on writes, diagnostics count of 2. Removing the early continue would increment the request counter and hang shutdown. A Dispatch fallthrough would also change the write count. Static prediction: would be detected.
The client keeps pending requests intact (response, cancel, EOF, disposal, request id 0) Null-id errors arrive while a request is pending. The request does not complete, and a later correlated response, cancel, disconnect or dispose still completes it correctly. RunTestsAsync_NullIdErrors_PreservePendingRequestLifecycle (5 cases), JsonRpcConnection_NullIdError_DoesNotCompleteRequestZero. Mapping a null id to 0 would complete or fail request 0: Assert.IsFalse(request.IsCompleted) would fail. Static prediction: would be detected.
End-to-end packed server and source client Real packed server and consumer, modern TFMs and net48, survive repeated null-id frames. Packaged acceptance tests in MtpServerClientPackagedConsumerRunTests. Reported as failing in the author's paired run. Executed evidence reported by the author but not accessible to me. This workflow does not claim it.
Docs and schema The null-id rule is documented and the schema is updated. Not executable. None. Not applicable. Schema validation is reported by the author as executed.

Remaining gaps: there is no accessible CI or TRX record bound to this head. The Rider and IDE recovery gap is noted by the author and not verifiable here.

🧵 Parallel safety

PARALLEL-SAFETY RESULT
STATUS: PARTIAL

The separate specialist task was not run in this execution, so parallel-safety coverage is incomplete. This is not a clean result. From the changed tests, I saw per-test state: a new FakeMtpServer, ConcurrentQueue, ServerFixture and MtpJsonRpcConnection per test, with no static or shared state added. The PR description states that no scheduling configuration changed, and no config candidates were in the manifest. Run /parallel-audit for a full assessment.

Per-test assessment

Pseudo-mutation evidence below consists of static predictions, not executed mutation results.

Result Quality Test Notes How to improve
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_NullIdError_PreservesUncorrelatedPayload Asserts id, code, message and data exactly for three codes. None
Pass A (90–100) new FormatterUtilitiesTests.Serialize_NullIdError_EmitsExplicitNullId Checks both the dictionary and the exact JSON output. None
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_CorrelatedError_PreservesIdKind Boundary ids with exact id and StringId expectations. None
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_NullIdError_RejectsInvalidEnvelope 13 malformed envelopes, each checked against both entry points. None
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_MissingErrorId_RemainsInvalid Exact exception type for a missing id. None
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_NullSuccessId_RemainsInvalid Keeps the null id error-only. None
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_NullCancellationId_RemainsInvalid Keeps the null id out of cancellation. None
Pass A (90–100) new ServerTestHostTests.RunAsync_NullIdErrorsAreDiagnosedWithoutLeakingRequestCounts Checks exit code, single reply, diagnostic count and logger failure. None
Pass A (90–100) new MtpServerClientTests.RunTestsAsync_NullIdErrors_PreservePendingRequestLifecycle Deterministic barrier, then each completion path asserted. None
Pass A (90–100) new MtpServerClientTests.JsonRpcConnection_NullIdError_DoesNotCompleteRequestZero Request id 0 stays pending until the correlated response. None
Uncertain — mod MtpServerClientPackagedConsumerRunTests (packaged null-id tests and helpers) Acceptance tests spawn packed servers and were not graded method by method. Supply the method bodies in scope, or run /review-tests with them.

No inline comments are needed: no verified, actionable defect passed the publication gate. Existing review threads: none.

<sub>This advisory review is heuristic and non-blocking.

Re-run the combined review with /review-tests, or only the parallel-safety specialist with /parallel-audit.</sub>

🤖 Automated content by GitHub Copilot. Generated by the Test Quality Review on PR (on open / sync) workflow. · copilot · auto · ~29 AIC · ⌖ ~1 AIC · ⊞ 8.1K

@microsoft-github-policy-service microsoft-github-policy-service Bot added state/needs-review Awaiting review from the team. and removed state/needs-review Awaiting review from the team. labels Nov 9, 2026
@github-actions github-actions Bot added state/needs-review Awaiting review from the team. and removed state/needs-review Awaiting review from the team. labels Nov 9, 2026
@microsoft-github-policy-service microsoft-github-policy-service Bot added state/needs-review Awaiting review from the team. and removed state/needs-review Awaiting review from the team. labels Nov 9, 2026
@github-actions github-actions Bot added state/approved Proposal approved; ready for implementation. and removed state/needs-review Awaiting review from the team. labels Nov 9, 2026
@microsoft-github-policy-service microsoft-github-policy-service Bot added the state/needs-review Awaiting review from the team. label Nov 9, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The review identifies no active issues requiring changes.

No open issues 🎉

What changed in this PR

Fixes #12261 by keeping Microsoft.Testing.Platform server-mode sessions alive after valid JSON-RPC errors with an explicit null ID, without correlating them to pending requests or changing other ID rules.

Changes:

  • Support uncorrelated errors in both JSON engines and the source-only client.
  • Diagnose these errors without replying, completing unrelated requests, or increasing shutdown work counts.
  • Update protocol documentation and add unit and packaged-consumer regression coverage, including .NET Framework.

Test-value evidence: The PR reports paired counterfactual failures and passing restored-implementation runs for revision 9ab03e7774cb99667362e88e0157dc2d10e89fb1. Execution artifacts were not independently verified here; real Rider recovery remains explicitly unverified.

File Description
test/​UnitTests/​Microsoft.Testing.Platform.UnitTests/​ServerMode/​FormatterUtilitiesTests.cs Tests null-ID serialization, payload preservation, and invalid envelopes.
test/​UnitTests/​Microsoft.Testing.Platform.UnitTests/​Hosts/​ServerTestHostTests.cs Tests diagnostics, logger failures, and balanced shutdown accounting.
test/​UnitTests/​Microsoft.Testing.Platform.ServerMode.Client.Sources.UnitTests/​MtpServerClientTests.cs Tests pending-request lifecycle and request-zero isolation.
test/​IntegrationTests/​Microsoft.Testing.Platform.Acceptance.IntegrationTests/​ServerMode/​v1.0.0/​TestingPlatformClient.cs Adds literal error-frame injection and process-output access.
test/​IntegrationTests/​Microsoft.Testing.Platform.Acceptance.IntegrationTests/​ServerMode/​ServerModeTestsBase.cs Supports additional fixtures, captures output, and stops listeners.
test/​IntegrationTests/​Microsoft.Testing.Platform.Acceptance.IntegrationTests/​MtpServerClientPackagedConsumerRunTests.cs Adds packaged client/server regressions and binary-provenance checks.
src/​Platform/​Microsoft.Testing.Platform/​ServerMode/​JsonRpc/​SerializerUtilities.RpcMessageSerializers.cs Preserves explicit null IDs in dictionary serialization.
src/​Platform/​Microsoft.Testing.Platform/​ServerMode/​JsonRpc/​SerializerUtilities.Deserializers.cs Accepts null-ID errors while validating their envelopes.
src/​Platform/​Microsoft.Testing.Platform/​ServerMode/​JsonRpc/​RpcMessages.cs Makes the error-message ID nullable.
src/​Platform/​Microsoft.Testing.Platform/​ServerMode/​JsonRpc/​Json/​Json.Serializers.cs Emits null error IDs in the modern JSON engine.
src/​Platform/​Microsoft.Testing.Platform/​ServerMode/​JsonRpc/​Json/​Json.Deserializers.cs Adds modern-engine parsing and validation for uncorrelated errors.
src/​Platform/​Microsoft.Testing.Platform/​InternalAPI/​InternalAPI.Unshipped.txt Records nullable internal error-message signatures.
src/​Platform/​Microsoft.Testing.Platform/​Hosts/​ServerTestHost.MessageLoop.cs Diagnoses uncorrelated errors before request accounting.
src/​Platform/​Microsoft.Testing.Platform.ServerMode.Client.Sources/​Client/​SerializerUtilities.ClientSerializers.cs Routes null-ID errors through client deserialization.
src/​Platform/​Microsoft.Testing.Platform.ServerMode.Client.Sources/​Client/​MtpJsonRpcConnection.cs Logs uncorrelated errors without completing pending requests.
docs/​mstest-runner-protocol/​server-mode-1.0.schema.json Allows null error IDs and excludes conflicting result payloads.
docs/​mstest-runner-protocol/​001-protocol-intro.md Documents uncorrelated errors and recovery limitations.

Review effort: Balanced

@github-actions github-actions Bot added state/approved Proposal approved; ready for implementation. and removed state/needs-review Awaiting review from the team. state/approved Proposal approved; ready for implementation. labels Nov 9, 2026
Address review feedback while preserving separate ID-presence, null-value, and exact JSON output checks. No product behavior changes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 9, 2026 13:29
@github-actions github-actions Bot added state/approved Proposal approved; ready for implementation. and removed state/approved Proposal approved; ready for implementation. labels Nov 9, 2026
@microsoft-github-policy-service microsoft-github-policy-service Bot added state/needs-review Awaiting review from the team. and removed state/needs-review Awaiting review from the team. labels Nov 9, 2026
@microsoft-github-policy-service microsoft-github-policy-service Bot added state/needs-review Awaiting review from the team. and removed state/needs-review Awaiting review from the team. labels Nov 9, 2026
@github-actions github-actions Bot added state/approved Proposal approved; ready for implementation. and removed state/needs-review Awaiting review from the team. state/approved Proposal approved; ready for implementation. labels Nov 9, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧪 Test quality review — PR #12283

Compared merge base 127798de9bd5c3f7204dc8932cddf043bf4bd6c6 with head efa13856c4dcdde6891fe844790f2d20c26aa1c8. Evidence applies to this head, not every intermediate commit. This review is static; no tests or builds were run. The only change since the previous review (9ab03e77) is a two-line lookup tweak in FormatterUtilitiesTests.

Test-value evidence

Contract Decisive test and assertion Counterfactual Evidence status
A null-id error is accepted as an uncorrelated ErrorMessage and keeps its payload (both JSON engines) Deserialize_NullIdError_PreservesUncorrelatedPayload: Assert.IsNull(error.Id) plus exact code, message and data The pre-fix strict-id deserializer throws MessageFormatException Static prediction: detected. The author reports an executed pair at 9ab03e77, but I could not access those records, so it is unverified here.
The serializer emits an explicit "id":null Serialize_NullIdError_EmitsExplicitNullId: exact JSON equality Reverting the (object?) cast or the nullable model drops or breaks the id Static prediction: detected
Envelope validation is otherwise unchanged Deserialize_NullIdError_RejectsInvalidEnvelope (13 cases), Deserialize_CorrelatedError_PreservesIdKind, Deserialize_MissingErrorId_RemainsInvalid, Deserialize_NullSuccessId_RemainsInvalid, Deserialize_NullCancellationId_RemainsInvalid Relaxing the id or envelope checks fails ThrowsExactly<MessageFormatException> Static prediction: detected
The host diagnoses a null-id error without replying or leaking the request counter RunAsync_NullIdErrorsAreDiagnosedWithoutLeakingRequestCounts: exit 0, a single write, 2 diagnostics, with and without a throwing logger Removing the early continue increments the counter and hangs shutdown Static prediction: detected
The client keeps pending requests intact (response, cancel, EOF, dispose, id 0) RunTestsAsync_NullIdErrors_PreservePendingRequestLifecycle (5 cases), JsonRpcConnection_NullIdError_DoesNotCompleteRequestZero Mapping a null id to 0 completes request 0, so Assert.IsFalse(request.IsCompleted) fails Static prediction: detected
End-to-end packed server and source client The packaged tests in MtpServerClientPackagedConsumerRunTests The author reports they failed without the fix Reported by the author only; no accessible record is bound to this head
Docs and schema Not executable None Not applicable

Remaining gap: no CI or TRX record is accessible for this head, and Rider recovery is unverified (the author notes it as a gap).

🧵 Parallel safety

STATUS: CLEAN. No findings (Critical 0, High 0, Warning 0, Info 0). The parallel-safety specialist's coverage was read-only, and it did not verify the worker count or scheduling attributes.

Test assembly Scope
Microsoft.Testing.Platform.Acceptance.IntegrationTests Method-level; no opt-out attribute
Microsoft.Testing.Platform.UnitTests and ServerMode.Client.Sources.UnitTests Changed unit tests

No parallelization configuration changed. The new tests use per-test fixtures and distinct asset names (-null-error-{tfm}), and add no static mutable state.

Per-test assessment

Pseudo-mutation evidence consists of static predictions, not executed mutation results.

Result Quality Test Notes How to improve
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_NullIdError_PreservesUncorrelatedPayload Asserts id, code, message and data exactly for three codes. None
Pass A (90–100) new FormatterUtilitiesTests.Serialize_NullIdError_EmitsExplicitNullId Checks both the dictionary and the exact JSON. None
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_CorrelatedError_PreservesIdKind Boundary ids with exact id and StringId expectations. None
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_NullIdError_RejectsInvalidEnvelope 13 malformed envelopes, each checked against both entry points. None
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_MissingErrorId_RemainsInvalid Exact exception type for a missing id. None
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_NullSuccessId_RemainsInvalid Keeps the null id error-only. None
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_NullCancellationId_RemainsInvalid Keeps the null id out of cancellation. None
Pass A (90–100) new ServerTestHostTests.RunAsync_NullIdErrorsAreDiagnosedWithoutLeakingRequestCounts Checks exit code, single reply, diagnostic count and logger failure. None
Pass A (90–100) new MtpServerClientTests.RunTestsAsync_NullIdErrors_PreservePendingRequestLifecycle A deterministic barrier, then each completion path is asserted. None
Pass A (90–100) new MtpServerClientTests.JsonRpcConnection_NullIdError_DoesNotCompleteRequestZero Request id 0 stays pending until the correlated response. None
Uncertain — mod MtpServerClientPackagedConsumerRunTests (packaged null-id tests and helpers) The process-spawning acceptance tests were not graded method by method. Supply the method bodies in scope, or re-run /review-tests with them.

No inline comments: no verified, actionable defect passed the publication gate. No live review threads exist on this PR.

<sub>This advisory review is heuristic and non-blocking.

Re-run the combined review with /review-tests, or only the parallel-safety specialist with /parallel-audit.</sub>

🤖 Automated content by GitHub Copilot. Generated by the Test Quality Review on PR (on open / sync) workflow. · copilot · auto · ~63 AIC · ⌖ ~1 AIC · ⊞ 8.1K

@microsoft-github-policy-service microsoft-github-policy-service Bot added the state/needs-review Awaiting review from the team. label Nov 9, 2026
@github-actions github-actions Bot added state/approved Proposal approved; ready for implementation. and removed state/needs-review Awaiting review from the team. state/approved Proposal approved; ready for implementation. labels Nov 9, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The review identifies no active issues with the null-ID JSON-RPC error-handling change.

No open issues 🎉

Review effort: Balanced

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Use the existing MTP_CLIENT_USE_MODERN_DOTNET package flag instead of NETCOREAPP in the generated null-error probe. Add a cross-platform net6 consumer case so the Jsonite fallback is exercised even when Unix execute-permission tests are unavailable.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 9, 2026 15:25
@github-actions github-actions Bot added state/approved Proposal approved; ready for implementation. and removed state/approved Proposal approved; ready for implementation. labels Nov 9, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧪 Test quality review — PR #12283

Compared merge base 127798de9bd5c3f7204dc8932cddf043bf4bd6c6 with head 11238b81d96408ae11646989250344ff8943ff43. Evidence applies to this head, not every intermediate commit.

Test-value evidence

All entries are static predictions. No paired execution records were available, and this workflow does not run tests.

Contract Scenario and expected outcome Test and decisive assertion Counterfactual Evidence
Null-id error deserialization (both serializers, client and server) {"id":null,"error":{...}} yields an ErrorMessage with Id == null and the payload preserved. FormatterUtilitiesTests.Deserialize_NullIdError_PreservesUncorrelatedPayload; Assert.IsNull(error.Id) The old code requires a non-null id and throws. Static prediction: detected.
Invalid null-id envelopes rejected result/method present, or a malformed error.code/error.message, throws MessageFormatException. Deserialize_NullIdError_RejectsInvalidEnvelope (13 cases); ThrowsExactly<MessageFormatException> Dropping the validation would accept these envelopes. Static prediction: detected.
A missing id stays invalid, and correlated int/string id kinds are preserved. A missing id throws. Int and string ids keep their kind. Deserialize_MissingErrorId_RemainsInvalid, Deserialize_CorrelatedError_PreservesIdKind Treating an absent id as null would break the first. Conflating id kinds would break the second. Static prediction: detected.
Null id serialized as an explicit null The serialized JSON contains "id":null. Serialize_NullIdError_EmitsExplicitNullId; exact JSON equality Omitting the id or using 0 fails the assertion. Static prediction: detected.
Server ignores uncorrelated errors Two null-id errors do not complete or leak request counts, the initialize response is written once, and the diagnostics are logged. This holds with a failing logger too. ServerTestHostTests.RunAsync_NullIdErrorsAreDiagnosedWithoutLeakingRequestCounts; diagnostics == 2, WriteRequestAsync Times.Once Without the guard, _requestCounter.AddCount() runs and the error is routed as id 0. Static prediction: detected.
Client keeps pending requests alive A null-id error must not complete request id 0 or any pending run. MtpServerClientTests.JsonRpcConnection_NullIdError_DoesNotCompleteRequestZero; request.IsCompleted is false and the pending count is 1 The old code mapped a null id to 0 and would fail the request. Static prediction: detected.
Client run lifecycle with null-id errors The run stays pending, then ends normally by response, cancellation, disconnect or disposal. RunTestsAsync_NullIdErrors_PreservePendingRequestLifecycle (5 cases) The same as the entry above. Static prediction: detected.
Docs, schema and acceptance consumers These are non-executable or integration-level changes. MtpServerClientPackagedConsumerRunTests, ServerModeTestsBase and TestingPlatformClient were not traced in depth. n/a Docs and schema: not applicable. Acceptance coverage: unverified.

🧵 Parallel safety

STATUS: CLEAN. The specialist's pass over the diff found no new shared static state, ports or environment mutation. The packaged-consumer tests use uniquely named assets. This was a read-only diff review, and parallelization settings were not determined. No findings.

Per-test assessment

Pseudo-mutation evidence below consists of static predictions, not executed mutation results. Production context was resolved from the diff, and the test-gap-analysis and grade-tests skill workflows were not run in full, so this scorecard is a manual assessment. The acceptance-test helper changes were not graded in detail.

Result Quality Test Notes How to improve
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_NullIdError_PreservesUncorrelatedPayload Asserts every field of the preserved payload. None
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_NullIdError_RejectsInvalidEnvelope 13 cases, each exercised through two entry points. None
Pass A (90–100) new FormatterUtilitiesTests.Serialize_NullIdError_EmitsExplicitNullId Checks both the property dictionary and the exact JSON. None
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_CorrelatedError_PreservesIdKind Boundary ids cover both id kinds. None
Pass A (90–100) new FormatterUtilitiesTests.Deserialize_MissingErrorId_RemainsInvalid Pins the negative contract. None
Pass A (90–100) new ServerTestHostTests.RunAsync_NullIdErrorsAreDiagnosedWithoutLeakingRequestCounts Uses barriers and a failing-logger case, with no wall-clock assertions. None
Pass A (90–100) new MtpServerClientTests.RunTestsAsync_NullIdErrors_PreservePendingRequestLifecycle Deterministic barrier, five lifecycle paths. None
Pass B (80–89) new MtpServerClientTests.JsonRpcConnection_NullIdError_DoesNotCompleteRequestZero Reflection on the private _nextRequestId couples it to the field name. None

No inline comments were published. Existing-thread deduplication was not needed, since no inline candidates passed the publication gate.

<sub>This advisory review is heuristic and non-blocking.

Re-run the combined review with /review-tests, or only the parallel-safety specialist with /parallel-audit.</sub>

🤖 Automated content by GitHub Copilot. Generated by the Test Quality Review on PR (on open / sync) workflow. · copilot · auto · ~54 AIC · ⌖ ~1 AIC · ⊞ 8.1K

@microsoft-github-policy-service microsoft-github-policy-service Bot added the state/needs-review Awaiting review from the team. label Nov 9, 2026
@github-actions github-actions Bot added state/approved Proposal approved; ready for implementation. and removed state/needs-review Awaiting review from the team. state/approved Proposal approved; ready for implementation. labels Nov 9, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the null-ID JSON-RPC error-handling change.

No open issues 🎉

Review effort: Balanced

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Preliminary test-triage analysis — Azure Pipelines build 1671113

Triggering check: microsoft.testfx (Build Windows Debug), job "Windows Debug", step "Test (unit tests only)". This is a partial, early analysis. Other build legs may still change the conclusion.

Failure signature

  • Test: MSTest.Analyzers.Test.AnalyzerProcessCompatibilityTests.WaitForExit_ReleasedOrAlreadyExitedChild_PreservesTheExitCode, reported as AnalyzerWaitForRunningProcess [net472].
  • Error: Assert.IsFalse(waiting.IsCompleted) failed with actual: true, at AnalyzerProcessCompatibilityTests.cs(41).
  • The assertion checks that the product Polyfill.WaitForExitAsync task is still pending while the child process is running. The task was already complete at that point.
  • Only this test failed. The step exited with code 2. The test run reports it once, as [attempt 1]. No later passing attempt is recorded, so there is no fail-then-pass retry evidence.

Assessment

  • Category: deterministic assertion failure, not an environmental failure. The only other signal is a low-memory warning (96% used) on the agent.
  • Confidence: medium-low. History is unavailable in early mode, so I can't say whether this is new or a flake. No crash or hang diagnostics were found.
  • I did not infer a root cause from the test name. The evidence only shows that the wait task completed early on net472 in this leg.

Next step
Check whether this test passes on net472 in the other legs and on the PR's base. Then review whether the product Polyfill.WaitForExitAsync can complete before the child exits on net472 (or whether the test's readiness gating is the issue). A final resolution comment will follow when the build completes.

🤖 Automated content by GitHub Copilot. Generated by the Pipeline Test Triage workflow. · copilot · auto · ~27 AIC · ⌖ ~1 AIC · ⊞ 10.2K

@Evangelink
Amaury Levé (Evangelink) merged commit 680411e into main Nov 9, 2026
28 of 30 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/rider-protocol-investigation branch November 9, 2026 18:40
@github-actions github-actions Bot removed the state/approved Proposal approved; ready for implementation. label Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Issue using Microsoft.Testing.Platform in Rider with TUnit

3 participants